从有限的资源中获得最大收益可以进步自然语言处理（NLP）研究和实践，同时保守资源。这些资源可能是数据，时间，存储或能源。NLP的最新工作从缩放率产生了有趣的结果。但是，仅使用比例来改善结果意味着资源消耗也会扩展。这种关系激发了对有效方法的研究，这些方法需要更少的资源才能获得相似的结果。这项调查涉及NLP效率的方法和发现，旨在指导该领域的新研究人员并激发新方法的发展。

临床表型可以从患者记录中自动提取临床状况，这可能对全球医生和诊所有益。但是，当前的最新模型主要适用于用英语编写的临床笔记。因此，我们研究了跨语化知识转移策略，以针对不使用英语并且有少量可用数据的诊所执行此任务。我们评估了希腊和西班牙诊所的这些策略，利用来自心脏病学，肿瘤学和ICU等不同临床领域的临床笔记。我们的结果揭示了两种策略，这些策略优于最先进的方法：基于翻译的方法，结合了域的编码器和跨语性编码器以及适配器。我们发现，这些策略在对稀有表型进行分类方面表现特别好，我们建议在哪种情况下更喜欢哪种方法。我们的结果表明，使用多语言数据总体可以改善临床表型模型，并可以补偿数据稀疏性。

基于临床票据的决策支持系统有可能通过指向监督风险的医生来改善患者护理。预测患者的结果是这种系统的重要组成部分，其中利用深神经网络的使用表明了有希望的结果。然而，这些网络学到的模式大多是不透明的，之前的工作揭示了关于非预期偏差的再现的缺陷。因此，我们引入了一个可扩展的测试框架，评估了关于输入变化的临床结果模型的行为。该框架有助于了解学习模式及其对模型决策的影响。在这项工作中，我们将其应用于对患者特征性别，年龄和种族的行为变化。我们对三个目前的临床NLP模型的评估表明了这些特征对模型决策的具体影响。他们表明，即使在相同的数据上微调并且据称最佳的模型并不总是学习最卓越的模式的模式，模型行为也变得剧烈变化。

智能建筑中的室内热舒适对乘员的健康和表现有重大影响。因此，机器学习（ML）越来越多地用于解决与室内热舒适的挑战。热舒适感的时间变化是调节居住者福祉和能耗的重要问题。但是，在大多数基于ML的热舒适研究中，不考虑时间中的时间方面，例如一天中的时间，昼夜节律和室外温度。这项工作解决了这些问题。它研究了昼夜节律和室外温度对ML模型的预测准确性和分类性能的影响。数据是通过在14个教室中进行的长达一个月的实地实验收集的，其中512名小学生。四个热舒适度指标被认为是深神经网络的输出，并支持数据集的向量机模型。时间变异性对学童舒适性的影响通过“一天中的时间”分析显示。预测准确性的时间差异已显示（多达80％）。此外，我们表明室外温度（随时间变化）对热舒适模型的预测性能产生了积极影响高达30％。时空环境的重要性通过对比的是微观级别（特定于位置）和宏观级别（整个城市的6个位置）的重要性。这项工作的最重要发现是，对于多种热舒适度指标，显示了预测准确性的明确提高，而天空中的时间和天空照明则有所增加。

多级优化已被广泛用作无数机器学习问题的数学基础，例如超参数优化，元学习和增强学习，仅举几例。尽管如此，实施多级优化程序通常需要在数学和编程方面的专业知识，这在该领域的研究都阻碍了研究。我们通过引入贝蒂（Betty）（用于基于梯度的多级优化的高级软件库）迈出了缩小这一差距的第一步。为此，我们基于对多级优化作为数据流图的新解释开发自动分化过程。我们进一步将多级优化的主要组成部分作为Python类，以实现简单，模块化和可维护的编程。我们从经验上证明，Betty可以用作一系列多级优化程序的高级编程接口，同时观察到测试准确性的提高11 \％，GPU存储器使用率下降14 \％，而20 \％降低了。在多个基准上的现有实现的墙壁时间。该代码可从http://github.com/leopard-ai/betty获得。

室内环境中的热舒适感会对乘员的健康，福祉和表现产生巨大影响。鉴于对能源效率和实现智能建筑的关注，机器学习（ML）越来越多地用于数据驱动的热舒适度（TC）预测。通常，提出了用于空调或HVAC通风建筑物的基于ML的解决方案，这些模型主要是为成年人设计的。另一方面，在大多数国家 /地区，自然通风（NV）的建筑物是常态。它们也是节能和长期可持续性目标的理想选择。但是，NV建筑物的室内环境缺乏热调节，并且在空间环境中差异很大。这些因素使TC预测极具挑战性。因此，确定建筑环境对TC模型性能的影响很重要。此外，需要研究跨不同NV室内空间的TC预测模型的概括能力。这项工作解决了这些问题。数据是通过在5个自然通风的学校建筑中进行的为期一个月的实地实验，涉及512名小学生。空间变异性对学生舒适度的影响通过预测准确性的变化（高达71％）来证明。还通过特征重要性的变化来证明建筑环境对TC预测的影响。此外，对儿童（我们的数据集）和成人（ASHRAE-II数据库）进行了模型性能的空间变异性比较分析。最后，评估了NV教室中热舒适模型的概括能力，并强调了主要挑战。

Modeling lies at the core of both the financial and the insurance industry for a wide variety of tasks. The rise and development of machine learning and deep learning models have created many opportunities to improve our modeling toolbox. Breakthroughs in these fields often come with the requirement of large amounts of data. Such large datasets are often not publicly available in finance and insurance, mainly due to privacy and ethics concerns. This lack of data is currently one of the main hurdles in developing better models. One possible option to alleviating this issue is generative modeling. Generative models are capable of simulating fake but realistic-looking data, also referred to as synthetic data, that can be shared more freely. Generative Adversarial Networks (GANs) is such a model that increases our capacity to fit very high-dimensional distributions of data. While research on GANs is an active topic in fields like computer vision, they have found limited adoption within the human sciences, like economics and insurance. Reason for this is that in these fields, most questions are inherently about identification of causal effects, while to this day neural networks, which are at the center of the GAN framework, focus mostly on high-dimensional correlations. In this paper we study the causal preservation capabilities of GANs and whether the produced synthetic data can reliably be used to answer causal questions. This is done by performing causal analyses on the synthetic data, produced by a GAN, with increasingly more lenient assumptions. We consider the cross-sectional case, the time series case and the case with a complete structural model. It is shown that in the simple cross-sectional scenario where correlation equals causation the GAN preserves causality, but that challenges arise for more advanced analyses.

We present the interpretable meta neural ordinary differential equation (iMODE) method to rapidly learn generalizable (i.e., not parameter-specific) dynamics from trajectories of multiple dynamical systems that vary in their physical parameters. The iMODE method learns meta-knowledge, the functional variations of the force field of dynamical system instances without knowing the physical parameters, by adopting a bi-level optimization framework: an outer level capturing the common force field form among studied dynamical system instances and an inner level adapting to individual system instances. A priori physical knowledge can be conveniently embedded in the neural network architecture as inductive bias, such as conservative force field and Euclidean symmetry. With the learned meta-knowledge, iMODE can model an unseen system within seconds, and inversely reveal knowledge on the physical parameters of a system, or as a Neural Gauge to "measure" the physical parameters of an unseen system with observed trajectories. We test the validity of the iMODE method on bistable, double pendulum, Van der Pol, Slinky, and reaction-diffusion systems.

We propose Hierarchical ProtoPNet: an interpretable network that explains its reasoning process by considering the hierarchical relationship between classes. Different from previous methods that explain their reasoning process by dissecting the input image and finding the prototypical parts responsible for the classification, we propose to explain the reasoning process for video action classification by dissecting the input video frames on multiple levels of the class hierarchy. The explanations leverage the hierarchy to deal with uncertainty, akin to human reasoning: When we observe water and human activity, but no definitive action it can be recognized as the water sports parent class. Only after observing a person swimming can we definitively refine it to the swimming action. Experiments on ActivityNet and UCF-101 show performance improvements while providing multi-level explanations.

Artificial intelligence (AI) in the form of deep learning bears promise for drug discovery and chemical biology, $\textit{e.g.}$, to predict protein structure and molecular bioactivity, plan organic synthesis, and design molecules $\textit{de novo}$. While most of the deep learning efforts in drug discovery have focused on ligand-based approaches, structure-based drug discovery has the potential to tackle unsolved challenges, such as affinity prediction for unexplored protein targets, binding-mechanism elucidation, and the rationalization of related chemical kinetic properties. Advances in deep learning methodologies and the availability of accurate predictions for protein tertiary structure advocate for a $\textit{renaissance}$ in structure-based approaches for drug discovery guided by AI. This review summarizes the most prominent algorithmic concepts in structure-based deep learning for drug discovery, and forecasts opportunities, applications, and challenges ahead.